Mean Absolute Difference (MAD) may be used as an index for video coding complexity in an H.264 rate control model. In conventional systems, MAD is predicted by a linear regression model using the actual MAD of the previous stored frames. For bitstreams coded with temporal and quality scalability, such as the Annex G extension of H.264 of Scalable Video Coding (SVC), it may be difficult to select which regressor should be used to achieve accurate prediction of the current MAD. In some situations, the inaccuracy of the MAD can lead to inappropriate selection of a Quantization Parameter (QP), resulting in a poor rate control performance.